Evaluating Multilingual Text Encoders for Unsupervised Cross-Lingual Retrieval

Pretrained multilingual text encoders based on neural Transformer architectures, such as BERT (mBERT) and XLM, have achieved strong performance a myriad of language understanding tasks. Consequently, they been adopted go-to paradigm for cross-lingual representation learning transfer, rendering word embeddings (CLWEs) effectively obsolete. However, questions remain to which extent this finding generalizes 1) unsupervised settings 2) ad-hoc IR (CLIR) Therefore, in work we present systematic empirical study focused the suitability state-of-the-art document sentence retrieval tasks across large number pairs. In contrast supervised understanding, our results indicate that document-level CLIR – setup with no relevance judgments IR-specific fine-tuning pretrained fail significantly outperform models CLWEs. For sentence-level CLIR, demonstrate can be achieved. peak is not met using general-purpose ‘off-the-shelf’, but rather relying their variants further specialized